Repository navigation
feat(ai, adapters): a ./models catalog in each provider package - #1887
Closed
tombeckenham wants to merge 5 commits into
Closed
tombeckenham wants to merge 5 commits into
tombeckenham wants to merge 5 commits into
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Apps can now read a provider's models at runtime from that provider's package, for example
import { models } from '@tanstack/ai-anthropic/models'. Each record has the context window, max output tokens, prices, input kinds, and reasoning levels.modelCost(model, usage)from@tanstack/aiprices one call. This PR supersedes #1772 and #1791.Stacked on #1308. Review only the commits above it. #1308 adds the modelschemas client and the native
model-meta.tssync. This PR adds the runtime catalogs on top.🎯 Changes
./modelssubpath in 11 provider packages. There are 13 catalogs and about 1,380 models.ai-openaialso has./azure-models, andai-cloudflarealso has./gateway-models.@tanstack/aiadds theModelRecordtype andmodelCost.modelCosttakes theTokenUsagefromonUsage, uses long-context tiers, and prices 1-hour cache writes at their own rate. It returnsundefinedwhen a price it needs is not known.scripts/sync-model-catalogs.tsmakes onelistModelscall (GET /v1/models?activity=chat&pricing=1) through@modelschemas/client, and writes each catalog file. Rows use the client'sModeltype. The client comes from feat(model-sync): sync native models from modelschemas only, with no prices in model metadata #1308'screateSyncClient, soMODELSCHEMAS_API_KEYapplies. It runs withnode. The daily sync workflow has a new step for it, and the sync writes a patch changeset for each package whose models changed.undefined: prices, a single rate, limits, input kinds, and reasoning levels. An unknown effort value, input kind, or price operator throws. Reasoning levels come from modelschemas' own level map, or from its effort list.docs/advanced/model-catalog.mdis for app developers who build a model picker or show the cost of a run. The prompt-caching page links to it, and the agent skills point to the subpaths.What changed from #1772 and #1791, as asked on #1772:
@tanstack/ai-modelspackage is gone. The data lives with each adapter, and the types andmodelCostlive in@tanstack/ai.ModelCompat, the wire and endpoint fields, the hand-kept lists (BORROW,ONLY,EXTRA_MODELS,MODEL_OVERRIDES), the parity test, and the 2x rule for 1-hour cache writes.generatedAt. A sync that finds no change writes nothing.sync-provider-models.tsmoved toscripts/model-sync/changeset.ts, so both syncs share one copy.createSyncClientfrom feat(model-sync): sync native models from modelschemas only, with no prices in model metadata #1308 moved toscripts/model-sync/client.ts. That file has no relative imports, so thenodecatalog sync can import it.modelschemas.tsre-exports it.@modelschemas/clientis^0.2.1(built JavaScript since Publish built JavaScript for client, codegen, vite, and the CLI schemas#445). ItslistModelsmaps the rows itself, so one feat(model-sync): sync native models from modelschemas only, with no prices in model metadata #1308 test now expects any error for a payload with nomodelsarray, not its own message.✅ Checklist
pnpm run test:pr, or these tests do not apply to this pull request.docs/for this change, or this change is not user-facing.pnpm changeset), or this PR does not change a published package.🚀 Release Impact
Testing
Commands run.
node scripts/sync-model-catalogs.tsthrough@modelschemas/client0.2.1: it refreshed 5 catalogs and skipped the 8 with no change. A second build from the same data compared equal to the formatted files, so a sync with no change writes nothing.vitest run scripts/model-sync: 9 files, 88 tests passed, including feat(model-sync): sync native models from modelschemas only, with no prices in model metadata #1308's. CI does not run this folder (test:maintaineronly).vitest run tests/model-catalog.test.tsinpackages/ai: 3 passed.tsc --noEmitinpackages/ai: no errors. Inai-openai,ai-openrouter,ai-cloudflare, andai-bedrock: no errors in the catalog files.oxlint --type-awareon the new source files: clean.pnpm test:prand the E2E suite. CI runs them. No E2E spec is added, because the catalog is static data plus one pure function with no wire behavior.Manual test.
node scripts/sync-model-catalogs.ts. Expect one line per catalog file. Only data that changed upstream since this commit shows ingit status.pnpm --filter @tanstack/ai test:lib -- model-catalog.packages/ai-anthropic/src/models.ts.claude-sonnet-4-5hascacheWrite1h: 6and no reasoning levels, because modelschemas lists onlymode: 'budget'for it.How this PR makes testing easy.
scripts/model-sync/model-catalog.test.tscovers the row rules: level map, efforts, unknown values that throw, prices from a price expression with tiers and currency, aliases, and repeated ids.packages/ai/tests/model-catalog.test.tscoversmodelCost: each token kind, the tier threshold, and unknown prices.Risk / rollback
Low. Every change is additive: new subpaths, one new function, and two new types. No adapter reads the catalog yet. To undo, revert the PR. Data gaps (for example 100 of 202 Bedrock models with no prices) are modelschemas gaps. Fix them upstream, and the next daily sync picks them up.
Public API change
Before
// No runtime model data. Prices and limits came from each provider's docs.After